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We derive entropic Bell inequalities from considering entropy Venn diagrams. These entropic in- 
equalities, akin to the Braunstein-Caves inequalities, are violated for a quantum mechanical Einstein- 
Podolsky-Rosen pair, which implies that the conditional entropies of Bell variables must be negative 
in this case. This suggests that the satisfaction of entropic Bell inequalities is equivalent to the 
non-negativity of conditional entropies as a necessary condition for separability. 
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The essence of Bell inequalities is related to Ein- 
stein's notion of "realism" ||: that an object has "objec- 
tive properties" whether they are measured or not. Bell 
inequalities, in their simplest form, reflect constraints on 
the statistics of any three local properties of a collection 
of objects. These constraints must be obeyed if the three 
properties can be independently known for each object. 
An intuitive discussion of Bell inequalities in this con- 
text is due to Wigner |H,[|. Consider a set of objects, 
each characterized by three two-valued (or dichotomic) 
properties a, b, and c. Then, grouping the objects as a 
function of two (out of the three) properties (for instance 
grouping together objects having property a but not 6), 
it is easy to build a simple inequality relating the number 
of objects in various groups defined by different pairs of 
properties. For example, 

n(a, not b) < n(a, not c) + n(not b, c) . (1) 

While such an inequality only refers to the simultaneous 
specification of any pair of properties, its satisfaction de- 
pends on the existence of a probability distribution for all 
three. Thus, even when the three properties can not be 
accessed at the same time (for whatever reason), Eq. ([]]) 
still holds provided that there exists such an objective de- 
scription of each object using three parameters a, b, and 
c; therefore, Eq. |IJ) provides a straightforward test of 
"local realism" (i.e., the combination of objectivity and 
locality) . As confirmed experimentally || , an inequality 
such as Eq. (Q) can be violated in quantum mechanics. 
It is the uncertainty principle (implying that the simul- 
taneous perfect knowledge of two conjugate observables 
is impossible) which is at the root of such a violation. 
Arguments similar to those above are used to derive the 
Bell inequalities |lj or their generalization, the Clauser- 
Horne-Shimony-Holt (CHSH) inequalities (7), and their 
violation can be traced back to the nonexistence of an 
underlying joint probability distribution involving incom- 
patible variables. 

The purpose of this paper is to show that the vio- 
lation of Bell inequalities in quantum mechanics is di- 
rectly connected to the existence of negative quantum 
entropies, a feature which is classically forbidden. We 
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have shown in previous work [§ that a consistent quan- 
tum information theory treating classical correlation and 
quantum entanglement on the same footing implies that 
conditional entropies can be negative. This purely quan- 
tum behavior can be traced back to the fact that the 
eigenvalues of a "conditional density matrix" can exceed 
one. (In contrast, the conditional probabilities in classi- 
cal information theory are always bounded by one, which 
implies the classical property that conditional entropies 
are non-negative.) Negative conditional entropies appear 
precisely in the case of quantum entanglement , for in- 
stance for an Einstein-Podolsky-Rosen (EPR) wavefunc- 
tion, which is the typical object of Bell-type experiments. 
As a consequence, it is natural to seek for a relation be- 
tween this non-classical feature and the violation of Bell 
inequalities, the standard evidence for the existence of 
quantum nonlocal correlations. To begin with, we de- 
rive an entropic Bell inequality that resembles the con- 
ventional one but involves mutual entropies rather than 
correlation coefficients. This entropic Bell inequality is 
related to the Braunstein-Caves information Bell inequal- 
ity |)| and implies Schumacher's triangle inequality for 
information distances JlO| . Unlike those, however, our 
inequality has a structure isomorphic to the conventional 
one, and has a simple geometric interpretation based on 
the ternary entropy diagram describing the Bell variables 
a, b, and c. Indeed, we show that the violation of our 
entropic Bell inequalities implies that one out of three 
conditional entropies describing abc must be negative, a 
feature that eliminates any classical description of the 
system. We show that these entropic Bell inequalities 
are violated when performing Bell-type measurements on 
EPR pairs, for example, but not necessarily at the same 
angles as the conventional Bell inequalities. Therefore, 
our entropic Bell inequalities provide another necessary 
condition for separability, distinct from the standard Bell 
inequalities. 

Consider two widely separated entangled systems in 
general, or, more specifically, a pair of spin- 1/2 particles 
in a singlet state (Bohm's jlll version of an EPR pair) 

l*) = 4(lU)-UT)) ■ (2) 
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Assume an observer, acting independently on each par- 
ticle, can measure the spin component of that particle 
along two possible orientations, for example with a Stern- 
Gerlach setup. Let the first observer either measure the 
z-component of one of the particles (and call this observ- 
able A and the outcome of the measurement a) or else 
the component along an axis making an angle 9 with the 
x-axis (observable B, with outcome b). Correspondingly, 
the second observer measures (on the second particle) 
either the z-component (observable A') or else the com- 
ponent making an angle cj> with the z-axis (observable 
C) ]l2| . Locality implies that the two distant observers 
have no influence on each other, i.e., the decision to make 
one of the two possible measurements on the first parti- 
cle does not affect the outcome of the measurement on 
the other particle. Indeed, it is known that the marginal 
statistics of the outcome of the spin measurement on the 
second particle, c for instance, is unchanged whether one 
measures A or B on the first particle. Let us now out- 
line a general derivation of conventional Bell inequalities 
(see, e.g., (|l3)). Consider 3 dichotomic random variables 
A, B, and C that represent properties of the system and 
can only take on the values +1 or —1 with equal proba- 
bility (1/2). For our purposes, they stand of course for 
the measured spin components (either up or down along 
the chosen axis), i.e., the Bell variables. (As A' is fully 
anticorrelated with A, we do not make use of it.) Any 
random set of outcomes a, b, and c must obey 

ab + ac — be < 1 (3) 

along with the two corresponding equations obtained by 
cyclic permutation (a — > b — » c). Indeed, the left-hand- 
side of Eq. (Q) is equal to 1 when a — b, while it is equal 
to — 1±2 when a — —b. Taking the average of Eq. (||) and 
its permutations yields the three basic Bell inequalities 

{ab) + (ac) - (be) < 1 (4) 
(ab) - (ac) + (be) < 1 (5) 
- (ab) + (ac) + (be) < 1 (6) 

relating the correlation coefficients between pairs of vari- 
ables. Eqs. (||) and (||) can be summarized in the form 
of the standard Bell inequality (l3) 

\{ab)-{ac)\ + {bc)<l. (7) 

The important point is that inequalities (||^) involve 
only the simultaneous specification of two (out of the 
three) random variables, although it is assumed that the 
three variables possess an element of reality, i.e., they can 
in principle be known at the same time, even if not in 
practice. In other words, it is assumed that there exists 
an underlying joint probability distribution p(a,b,c), in 
which case the Bell inequalities (which depend only on 
the marginal probability p(a, b) = ^2 c p(a, b, c) and cyclic 
permutations) must be satisfied. Therefore, the violation 



of any of the inequalities (|[|6|) implies that a, b, and c 
cannot derive from a joint distribution (i.e., cannot be 
described by any local hidden- variable theory), as em- 
phasized in Ref. ||. In the following, we will show that 
the violation of Bell inequalities, while ruling out such a 
classical underlying description of local realism, still does 
not contradict a quantum one based on an underlying 
joint density matrix pabc, but forces the corresponding 
entropies to be negative || . 

Let us derive Bell inequalities akin to the conventional 
ones, Eqs. (^-^), but relating entropies for the three di- 
chotomic random variables A, B, and C. We assume 
that one has access to the entropy of each variable H (A), 
H(B), H(C), as well as to the mutual entropy between 
each pair of variables H(A:B), H(A:C), and H{B:C). 
Here, the entropies are Shannon entropies []l4j| , given by 

H(A) = -5>(a)logp(a) (8) 

a 

and the mutual entropies are defined by 

H(A:B) = H(A) + H(B) — H(AB) . (9) 

The mutual entropy H(A:B) corresponds to the entropy 
shared by A and B, or in other words to the information 
about A that is conveyed by B (or conversely). Physi- 
cally, H(A:B) is closely related to the correlation coeffi- 
cient between a and b. To establish notation, let us also 
define the conditional entropy H(A\B) as the entropy of 
variable A while "knowing", i.e., having measured, B, 

H(A\B)=H(AB)-H{B), (10) 

which allows us to separate any entropy into a conditional 
and a mutual piece with respect to another variable [fl5| : 

H(A) = H{A\B) + H(A:B) . (11) 

For a three- variable system we can split information into 
conditional and mutual information in the same fashion: 
the information H(A:B), for example, can be split as 

H(A:B) = H(A:B\C) + H(A:B:C) . (12) 

Thus, a conditional information such as H(A:B\C) = 
H(AC) + H(BC) - H(C) - H(ABC) is that piece of an 
information (between two variables) that is not shared by 
a third variable, i.e., the information conditional on the 
third variable. The piece of information that is shared 
by a third variable can be written as 

H(A:B:C) = H(A) + H(B) + H(C) 

- H(AB) - H(AC) — H(BC) + H(ABC) . (13) 

Let us now construct Bell inequalities involving only in- 
formations between pairs of variables (rather than corre- 
lation coefficients). Relations between entropies are con- 
veniently represented by entropy Venn diagrams M-tnl , 
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and inequalities can easily be read off them. As shown in 
Fig. [j], the ternary entropy diagram for the Bell vari- 
ables ABC has 7 (2™ - 1 with n = 3) entries. We 
use the symbols a, /?, 7 for conditional entropies [e.g., 
a = H(A\BC)], a, f3, 7 for conditional informations 
[e.g., a = H(B:C\A)}, and denote by 6 = H(A:B:C) 
the mutual information between the three Bell variables. 
Apart from the marginal statistics of each of the vari- 
ables A, B, and C, experimentally we also have access 
to the the marginal statistics of any pair (AB, AC, or 
BC), yielding six constraints. Consequently, we do not 
have enough constraints to completely fill in the entropy 
diagram of Fig. [I]: the missing constraint concerns the 
intrinsic three-body correlation which is not fixed by two- 
body statistics. The seven entries in the ternary entropy 
diagram can thus be expressed as a function of the six en- 
tropies H(A), H(B), H(C), H(A:B), H(A:C), H(B:C), 
plus a parameter 6, the inaccessible ternary mutual in- 
formation. 

FIG. 1. Ternary entropy diagram for the Bell variables 
ABC. The entries a, a, {3, /3, 7, 7 and S are defined in the 
text. All of them (except 5) are non- negative in Shannon 
information theory llq ]. 
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Despite this indeterminacy, the entries can be com- 
bined to give expressions independent of 5, and which 
therefore can be expressed in terms of measurable en- 
tropies only. More precisely, we find 

a + a = H(A) + H(B:C) - H(A:B) - H{A:C) , (14) 
/3 + /? = H{B) + H(A:C) - H(A:B) - H{B:C) , (15) 
7 + 7 = H{C) + H(A:B) - H{A:C) - H{B:C) . (16) 

If A, B and C describe a classical system, it is known 
that all the entries except S are non-negative. Indeed, 
monotonicity of Shannon entropies implies that condi- 
tional entropies such as a = H{A\BC) are positive semi- 
definite By the same token, conditional informa- 
tions such as a = H(B:C\A), as they describe informa- 
tion between two variables when a third is known, are 
non-negative. (This property is called strong subaddi- 
tivity p8|.) The indeterminacy of 5 can be traced back 
to the freedom in the choice of a local hidden-variable 
model to describe the marginal statistics, but its value 
is unimportant as far as questions of locality are con- 



cerned. From Eqs. ( pT| - |l6| ) it follows straightforwardly 
that the three inequalities 

H(A:B) + H{A:C)-H{B:C)<H(A) (17) 
H{A:B)-H{A:C)+H{B:C)<H{B) (18) 
-H(A:B) + H{A:C) + H{B:C)<H(C) (19) 

must be satisfied if the system ABC is classical. These 
equations therefore constitute entropic Bell inequalities. 
Note that in the case where A, B, and C have a uniform 
distribution, one has H(A) = H(B) = H(C) = 1; the in- 
equalities then become very similar to the standard ones 
[Eqs. (UH)], but relating mutual entropies rather than 
correlation coefficients. For instance, one can write 

\H(A:B) - H{A:C)\ + H(B:C) < 1 (20) 

from Eqs. (|l8| ) and (|l9|), in perfect analogy with Eq. (0). 
More generally, the CHSH inequalities for mutual en- 
tropies can be derived using the chain rule for entropies. 
The resulting inequality 

H{A':B) + H(A:C) - H(B:C) + H{A:A r ) < 2 , (21) 

is similar in form to the traditional CHSH inequality 
(see Jl3l), and implies the Braunstein-Caves inequality 0] 
as well as Schumacher's quadrilateral inequality |l0[ . 

The converse of the previous reasoning is most inter- 
esting. If the data that are extracted from marginal 
statistics show that one of the three entropic inequalities 
is violated, it implies that one of the three inequalities 
a + a > (etc.) is violated. Therefore, since strong 
subadditivity of quantum entropies |l8j implies that a, 
[3, and 7 are always > 0, one of the conditional entropies 
a, (3, or 7 must be negative, which of course is classically 
forbidden. Thus, a violation of an entropic Bell inequal- 
ity always goes hand in hand with the appearance of a 
negative conditional entropy in Fig. ^. This is the case for 
example in Bell measurements of EPR pairs, as we show 
in more detail below. Therefore, negative entropies au- 
tomatically rule out a description of the system in terms 
of local hidden variables (or an underlying joint proba- 
bility distribution). If there cannot be any such descrip- 
tion, it is well-known that the system in question is non- 
separable p9[| . Equivalently, it is shown in Refs. [|s||T6|l 
that the concavity of conditional quantum entropies im- 
plies that any separable density matrix is characterized 
by non-negative conditional entropies (see also po| ). In 
summary, the satisfaction of entropic Bell- inequalities, 
or equivalently the non-negativity of the corresponding 
entropies, is a necessary condition for separability, albeit 
not a sufficient one. Let us show that this condition is dis- 
tinct from the one based on the satisfaction of traditional 
Bell inequalities by considering as an example Bell exper- 
iments on EPR pairs. In this case, because the outcomes 
±1 occur with equal probability (1/2), the correlation 
coefficient can be written as (ab) = 4p++ — 1 = 1 — 4p_| — , 
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with p ++ ) being the probability to observe aligned 

(anti-aligned) spins. The mutual entropy (in bits) can 
then trivially be expressed in terms of the corresponding 
correlation coefficient via 



(22) 



Using the standard quantum results for the correlation 
coefficients, i.e., (ab) — —(a'b) — cos(0), (ac) = — cos(0), 
and (be) = — cos(0 — 4>), we plot in Fig. ||a the left-hand 
side of Eqs. ( |l7j - |l9| ) as a function of 4> for the "most vio- 
lating" angle 9 = 7r/3.958 (the maximum violation occurs 
at (f) = 9/2). Note that the conventional Bell inequali- 
ties Eqs. (^|||) are maximally violated at a different angle 
9 = 7r/3. Nevertheless, we have plotted the left-hand side 
of these equations at the same angle 9 as the entropic 
ones for comparison in Fig. ^|b. Despite the similarity in 
the structure of the equations, the violation of one con- 
ventional Bell inequality does not necessarily imply the 
violation of an entropic one, or vice versa. 

FIG. 2. fa) Left-hand side Le of entropic Bell inequalities 
Eqs. for EPR Bell-measurements with 6 = tt/3.958. 

The inequalities are violated if Le > 1; (b) Left-hand side Lc 
of conventional inequalities Eqs. (Q-^) at the same angle. 



(b) 




We have derived entropic Bell inequalities by demand- 
ing that the conditional entropies arising in the ternary 
entropy diagram for Bell variables be non-negative, pro- 
viding a necessary condition for separability. The exper- 
imental violation of Bell inequalities, traditionally inter- 
preted as ruling out the existence of a joint probability 
Pabc, therefore also reflects the appearance of negative 
conditional entropies in Bell-type measurements. In fact, 
these experiments do not rule out a description in terms 
of an underlying joint density matrix pabc- Yet, the lat- 
ter does not describe three physical systems as the EPR 



experiment only involves two detectors. Because of the 
degree of freedom involved with the choice of S, such a 
Pabc cannot be constructed explicitly. We are therefore 
uncertain as to the physical interpretation of Pabc-, & dif- 
ficulty inherent to independent Bell-type measurements 
on identically prepared systems. It has recently been 
suggested that consecutive measurements performed on a 
single quantum system are more apt at revealing "hidden 
nonlocality" |2l]]. It might therefore prove to be fruitful 
to apply the present analysis to such situations. 
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